A “Publish/Subscribe” system is a type of messaging application in which the providers of information (publishers) are decoupled from the consumers of that information (subscribers) by means of a broker. Typically, in such a system, a number of publishers publish messages to a broker on particular topics (e.g. news, weather, sport). Subscribers register their interest in such topics via subscription requests received at the broker. For example, a subscriber may request to receive any information published on the weather, whilst another subscriber may desire information on the news and sport. Publishers are not concerned with where their publications are going and subscribers are not interested in where the messages they receive have come from. Instead, the broker manages the distribution of the messages to make sure that they arrive at the correct destination according to the valid subscriber/topic pairs registered in a subscription list held by the broker. The broker also ensures that messages are distributed in the correct format and validates the authority of each publisher to publish to the subscribers which have subscribed to the particular topic encompassing the message.
In general terms, a publisher generates a message that it wants to publish and defines the topic of the message. The broker retrieves the message from its input node and passes it to a publication node for distribution to all subscribers that have registered an interest. Distribution of messages to subscribers may either be by point-to-point broadcast from the broker or may be by multi-casting. In the latter case, messages on particular topics are distributed to intermediate shared addresses which are provided to all the subscribers that have registered an interest in the particular topic so that they can listen in for newly published information.
When subscribers no longer have an interest in a given topic, they issue an “unsubscribe” request to the broker which then removes the subscriber/topic pair from its subscription list.
In larger systems, multiple brokers constitute the nodes of a larger, heterogeneous network. Topic information from publishers may enter the system at any of the brokers and be passed onward to other brokers through communication channels between the nodes of the system. As in the case of a single broker, each broker has its own subscribers which subscribe to it for information published on one or more topics defined in the subscriptions. These are registered in a subscription list, which lists local subscribers with currently active subscriptions and the topics to which they subscribe.
In multi-broker publish/subscribe systems, two distinct and complementary methodologies are known for dealing with publications and subscriptions, as passed between brokers. These are known as “publish-everywhere” and “proxy-subscription forwarding”. In the publish-everywhere methodology, publications and subscriptions are broadcast to every broker at every node in the network. In proxy subscription forwarding, on the other hand, a broker indicates to neighbouring brokers that it has subscribers for a particular topic. The neighbouring brokers do not know the identity of these subscribers but add the name of the forwarding broker and the topic concerned as a matched pair to their subscription lists along with their own local subscriptions. Thus, when data on the requested topic is published to the neighbouring broker, it knows to send the data on to the forwarding broker, as well as to its local subscribers, but does not send it to other brokers, not in its subscription list. Thus, proxy subscriptions may be regarded as filters for required topics.
Both of these approaches have different advantages and disadvantages which depend on the particular traffic throughput and the demands of both the publishers and subscribers that are active at a particular time. A fixed and specific choice between the two approaches may not always be optimal for a given configuration as it is not able to change to cope with the continuously varying demands of both publishers and subscribers. The disadvantages of each choice are as follows: If proxy-subscription forwarding is used then fast changing subscriptions cause large volumes of messages creating and deleting proxy-subscriptions to propagate throughout the whole network. This occurs particularly when a subscriber repeatedly unsubscribes and resubscribes to a topic. On the other hand, if a publish-everywhere approach is chosen then there will more than likely be large numbers of unwanted messages delivered to destinations that have no actual subscribers. Both techniques will thus waste network bandwidth in certain circumstances.
Published US Patent Application 2003/0135556 (S. Holdsworth et al.), assigned to IBM Corporation, shows the use of both broadcast and of proxy subscription filtering in a network of message brokers. Filtering policy between two or more brokers can be switched whenever network communication characteristics show that it would be more efficient to use broadcast than proxy forwarding or vice versa. A switching threshold between the two policies may be determined by measuring such network characteristics as the frequency of subscribe and unsubscribe requests or a count of the number of redundant messages. Another alternative is to look at delivery delays between brokers and their causes. Generally speaking, a dynamic switching policy is determined based on the set of messages being sent by one broker to another. If one broker decides to change policy, it informs the other of its decision.
The solution proposed by US 2003/0135556 is effective in reducing network traffic but does require complex knowledge of and measurements to be made on at least two brokers and the communication channel between them. It does not offer a stand alone solution, self contained within one broker. It also requires an explicit switch between broadcast and proxy subscription policies on a given threshold.